Display Device

ABSTRACT

A display device that is to be connected with an information terminal includes: a video reception unit that receives video information about a screen including a cursor from the information terminal; a touch panel that brings up on display the screen based upon the video information received by the video reception unit and detects a touch position specified by a touch operation; and a transmission unit that transmits, to the information terminal, displacement quantity information corresponding to a quantity of displacement from the display position of the cursor to the touch position.

TECHNICAL FIELD

The present invention relates to a display device.

BACKGROUND ART

There has been known in the related art a technology which makes itpossible to bring up on display a display screen of a portable phone anda plurality of touch switches corresponding to respective operation keysof the portable phone on the display of an in-vehicle device andoperate, when any particular touch switch among the touch switches isoperated, the portable phone by providing an operation command for anoperation key that corresponds to the particular touch switch from thein-vehicle device to the portable phone (see Patent Literature 1).

CITATION LIST Patent Literature

Patent Literature 1: Japanese Laid Open Patent Publication No.2003-244343

SUMMARY OF INVENTION Technical Problem

In recent years, in addition to conventional portable phones in whichoperations are performed by using operation keys, smartphones thatenable intuitive operations by using a touch panel are widely used. Itis impossible to achieve operations of such smartphones, through thein-vehicle device according to the conventional technology as describedin the patent literature 1. As stated above, conventionally, there hasbeen proposed no technology for operating an information terminal, suchas a smartphone, in which a touch panel input operation method isadopted, through a display device when it brings up on display a displayscreen of the information terminal thereon.

Solution to Problem

A display device according to a first aspect of the present invention isto be connected with an information terminal and it includes: a videoreception unit that receives video information about a screen includinga cursor from the information terminal; a touch panel that brings up ondisplay the screen based upon the video information received by thevideo reception unit and detects a touch position specified by a touchoperation; and a transmission unit that transmits, to the informationterminal, displacement quantity information corresponding to a quantityof displacement from the display position of the cursor to the touchposition.

According to a second aspect of the present invention, in the displaydevice of the first aspect, it is preferred that the transmission unitadditionally transmits button operation information indicating that apredetermined button operation has been performed to the informationterminal.

According to a third aspect of the present invention, in the displaydevice of the second aspect, it is preferred that, if the touchoperation is completed within a predetermined period of time with thetouch position being unchanged, the transmission unit transmits, afterhaving transmitted the displacement quantity information incorrespondence to the quantity of displacement from the display positionof the cursor to the touch position, the button operation informationalong with displacement quantity information indicating that thequantity of displacement is 0.

According to a fourth aspect of the present invention, in the displaydevice of the second or third aspect, it is preferred that, if the touchoperation is performed a plurality of times with the touch positionbeing unchanged, the transmission unit transmits, after havingtransmitted the displacement quantity information in correspondence tothe quantity of displacement from the display position of the cursor tothe touch position, the button operation information a plurality oftimes along with the displacement quantity information indicating thatthe quantity of displacement is 0.

According to a fifth aspect of the present invention, in the displaydevice of any one of the second through fourth aspects, it is preferredthat, if the touch position is changed while the touch operation isgoing on, the transmission unit transmits, after having transmitted thedisplacement quantity information in correspondence to the quantity ofdisplacement from the display position of the cursor to the touchposition, the button operation information along with the displacementquantity information indicating a quantity of displacement incorrespondence to a change in the touch position while the touchoperation is going on, and after the touch operation is completed,transmits displacement quantity information indicating that the quantityof displacement is 0.

According to a sixth aspect of the present invention, in the displaydevice of any one of the second through fifth aspects, it is preferredthat, if the touch operation is going on for a predetermined period oftime or longer with the touch position being unchanged, the transmissionunit transmits, after having transmitted the displacement quantityinformation in correspondence to the quantity of displacement from thedisplay position of the cursor to the touch position, the buttonoperation information along with displacement quantity informationindicating that the quantity of displacement is 0 while the touchoperation is going on.

According to a seventh aspect of the present invention, in the displaydevice of any one of the first through sixth aspects, it is preferredthat the transmission unit transmits the displacement quantityinformation by using a communication format for a mouse in Bluetooth orUSB.

A display device according to an eighth aspect of the present inventionis to be connected with an information terminal having a display monitorand it includes: a video reception unit that receives video informationabout a screen including a cursor from the information terminal; a touchpanel that brings up on display the screen based upon the videoinformation received by the video reception unit and detects a touchposition specified by a touch operation; a resolution informationreception unit that receives resolution information of the displaymonitor from the information terminal; a conversion coefficientcalculation unit that calculates a conversion coefficient incorrespondence to a ratio of resolution of the display monitor toresolution of the touch panel based upon the resolution informationreceived by the resolution information reception unit and resolution ofthe touch panel that is stored in advance; a displacement quantitycalculation unit that calculates a quantity of displacement from adisplay position of the cursor to the touch position; a conversion unitthat converts the quantity of displacement calculated by thedisplacement quantity calculation unit into a quantity of displacementon the display monitor based upon the conversion coefficient calculatedby the conversion coefficient calculation unit; and a transmission unitthat transmits, to the information terminal, displacement quantityinformation indicating the quantity of displacement converted by theconversion unit.

According to a ninth aspect of the present invention, in the displaydevice of the eighth aspect, it is preferred that: the conversioncoefficient calculation unit calculates the conversion coefficient for atransverse direction and a longitudinal direction, respectively, of thetouch panel; the displacement quantity calculation unit calculates aquantity of displacement from a displacement position of the cursor tothe touch position for the transverse direction and the longitudinaldirection, respectively, of the touch panel; and the conversion unitconverts the quantity of displacement for the transverse direction ofthe touch panel into a quantity of displacement for a transversedirection on the display monitor by using the conversion coefficient inthe transverse direction and also converts the quantity of displacementfor the longitudinal direction of the touch panel into a quantity ofdisplacement for a longitudinal direction on the display monitor byusing the conversion coefficient in the longitudinal direction.

A display device according to a tenth aspect of the present invention isto be connected with an information terminal and it includes: a videoreception unit that receives video information about a screen includinga cursor brought up on display at a predetermined reference positionfrom the information terminal; a touch panel that brings up on displaythe screen based upon the video information received by the videoreception unit and detects a touch position specified by a touchoperation; and a transmission unit that transmits, to the informationterminal, displacement quantity information corresponding to a quantityof displacement from the display position of the cursor to the touchposition. This display device returns, after completion of the touchoperation, the display position of the cursor to the reference position.

According to an eleventh aspect of the present invention, in the displaydevice of the tenth aspect, it is preferred that, after completion ofthe touch operation, the transmission unit transmits displacementquantity information indicating a predetermined maximal quantity ofdisplacement or a predetermined minimal quantity of displacement for thetransverse direction and the longitudinal direction, respectively, ofthe touch panel to the information terminal, thereby returning thedisplay position of the cursor to the reference position.

According to a twelfth aspect of the present invention, in the displaydevice of the tenth or eleventh aspect, it is preferred that: a point ofindication of the cursor is located at an end of the cursor; and thereference position is located at an end in an orientation opposite to anorientation directed from a center of the cursor to the point ofindication.

According to a thirteenth aspect of the present invention, in thedisplay device of the twelfth aspect, it is preferred that: the point ofindication of the cursor is located at a left upper end of the cursor;and the reference position is located at a right lower end of the touchpanel.

A display device according to a fourteenth aspect of the presentinvention is to be connected with an information terminal and itincludes: a video reception unit that receives an image from theinformation terminal; and a touch panel that brings up on display theimage received by the video reception unit and detects a touch positionspecified by a touch operation. This display device displaces, inresponse to the touch operation, a cursor brought up on display at anouter periphery of the touch panel to the touch position and, aftercompletion of the touch operation, returns the cursor to an originalposition.

According to a fifteenth aspect of the present invention, in the displaydevice of the fourteenth aspect, it is preferred that, after completionof the touch operation, displacement quantity information indicating apredetermined maximal quantity of displacement or a predeterminedminimal quantity of displacement for the transverse direction and thelongitudinal direction, respectively, of the touch panel is transmittedto the information terminal, thereby returning the display position ofthe cursor to the reference position.

According to a sixteenth aspect of the present invention, in the displaydevice of the fourteenth or fifteenth aspect, it is preferred that: apoint of indication of the cursor is located at a left upper end of thecursor; and the cursor is brought up on display at a right lower end ofthe touch panel before the touch operation is started and after thetouch operation is completed.

Advantageous Effect of the Invention

According to the present invention, when a display screen of aninformation terminal, in which a touch panel input operation method isadopted, is brought up on display on a display device, it is possible tooperate the information terminal through the display device.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 A diagram showing the configuration of the in-vehicle informationsystem achieved in an embodiment of the present invention.

FIG. 2 A block diagram showing the structures adopted in the in-vehicledevice and the portable terminal.

FIG. 3 A schematic diagram illustrating the software structure adoptedin the portable terminal.

FIG. 4 A diagram illustrating the behaviors of the in-vehicle device andthe portable terminal, respectively, when a touch operation is performedat the in-vehicle device.

FIG. 5 A flowchart illustrating the processing executed in thein-vehicle device when the touch operation is performed.

DESCRIPTION OF EMBODIMENTS

FIG. 1 shows the configuration of the in-vehicle information systemachieved in an embodiment of the present invention. The in-vehicleinformation system in FIG. 1, which is used as an onboard systeminstalled in a vehicle, is configured by connecting an in-vehicle device1 with a portable terminal 2 through short-range wireless communicationand through wired communication enabled via a video/audio cable 3. Thein-vehicle device 1 is installed at a fixed position within the vehicle,at, for instance, the instrument panel in the vehicle. The portableterminal 2 is a portable information terminal that can be carried by theuser, such as a portable telephone or a smart phone. It is to be notedthat short-range wireless communication between the in-vehicle device 1and the portable terminal 2 may be carried out in compliance with, forinstance, the Bluetooth standard. In addition, the wired communicationvia the video/audio cable 3 may be achieved in compliance with, forinstance, the HDMI (high definition multimedia interface) standard.

A display unit 11 is provided to the in-vehicle device 1. The displayunit 11 is a touch panel that is capable of displaying images and videoimages of various types, and, for example, may consist of a combinationof resistive film type touch panel switches and a liquid crystaldisplay. By performing touch operation with his finger or the like atany desired position on the display unit 11, and by thus designating anicon or operation button or the like displayed in that position, theuser is able to cause the portable terminal 2 to execute variousfunctions as desired. It should be understood that, in addition to thedisplay unit 11, it would also be acceptable further to provide thein-vehicle device 1 with various operation switches corresponding topredetermined operations.

A display unit 21 is provided to the portable terminal 2. The displayunit 21 is a touch panel that is capable of displaying images and videoimages of various types, and may, for example, consist of a combinationof capacitance type touch panel switches and a liquid crystal display.By touching any desired position on the display unit 21, according tothe details of the images or video images being displayed on the displayunit 21, the user is able to cause the portable terminal 2 to executevarious functions as desired. It is to be noted that while the displayunit 21 in this example is a touch panel, the present invention may beadopted in conjunction with a standard display monitor instead of atouch panel. In such a case, it is desirable that the portable terminal2 include various types of operation switches corresponding to specificcontents or details of processing executed by the portable terminal 2.As an alternative, the present invention may be adopted in conjunctionwith a display unit 21 constituted with a touch panel-type displaymonitor in an portable terminal 2 that also includes operation switches,each corresponding to a specific operation.

FIG. 2 is a block diagram showing the structures of the in-vehicledevice 1 and the portable terminal 2. As shown in FIG. 2, the in-vehicledevice 1 includes a control unit 10, the display unit 11, an operationunit 12, an audio output unit 13, a memory unit 14, a short-rangewireless communication interface unit 15 and a video/audio signal inputunit 16. The portable terminal 2 includes a control unit 20, the displayunit 21, an operation unit 22, an audio output unit 23, a memory unit24, a short-range wireless communication interface unit 25, avideo/audio signal output unit 26, a wireless communication unit 27 anda GPS (global positioning system) receiver unit 28.

The control unit 10 in the in-vehicle device 1, which is constitutedwith a microprocessor, various peripheral circuits, a RAM, a ROM and thelike, executes various types of processing based upon a control programrecorded in the memory unit 14. Various types of image displayprocessing, audio output processing and the like are enabled throughprocessing executed by the control unit 10.

In addition, the control unit 10 obtains a vehicle speed signal and aparking signal output from the vehicle. The control unit 10 determineswhether the vehicle is currently in a traveling state or in a stationarystate based upon the vehicle speed signal and the parking signal. It isto be noted that the vehicle speed signal and the parking signal may beprovided from the vehicle to the control unit 10 in the form of avehicle speed pulse output via a CAN (controller area network)configuring an onboard communication network, from a vehicle speedsensor installed in the vehicle.

As previously described with reference to FIG. 1, the display unit 11 isa display monitor that may consist of a liquid crystal display or thelike. The operation unit 12 is a section for detecting touch operationby the user on the display unit 11, and corresponds to the touch panelswitches described above. It should be understood that though thedisplay unit 11 and the operation unit 12 are shown separately in FIG.2, actually these are integrated together and constitute a touch panel.Moreover, if operation switches are provided to the in-vehicle device 1as previously described, then these operation switches are also includedin the operation unit 12. The information of input operations that theuser performs on the operation unit 12 is output to the control unit 10,and is reflected in the processing performed by the control unit 10.

The audio output unit 13, which includes an amplifier, a speaker and thelike, is capable of outputting various types of sound under controlexecuted by the control unit 10. For instance, music played back byreproducing music data read out from the portable terminal 2 or arecording medium (not shown), audio guidance used to guide the vehicleto a destination, or the like can be output from the audio output unit13.

The memory unit 14, which is a non-volatile data storage device, may beconstituted with an HDD (hard disk drive), a flash memory or the like.Various types of data, including the control program used by the controlunit 10, as explained earlier, are stored in the memory unit 14. Dataare read out from the memory unit 14 and written into the memory unit 14as needed under control executed by the control unit 10.

The short-range wireless communication interface unit 15 executes, undercontrol executed by the control unit 10, wireless interface processingrequired to enable short-range wireless communication with the portableterminal 2. For instance, it converts information output from thecontrol unit 10 to a wireless signal in a predetermined signal formatand transmits the wireless signal to the portable terminal 2. It alsoreceives information output from the portable terminal 2 as a wirelesssignal in a predetermined signal format and outputs the receivedwireless signal information to the control unit 10. The short-rangewireless communication interface unit 15 executes such interfaceprocessing in compliance with a specific communication standard such asBluetooth.

The video/audio signal input unit 16 converts a video signal and anaudio signal input from the portable terminal 2 via the video/audiocable 3 to image (video image) data for screen display and audio datafor audio output respectively and outputs them to the control unit 10.The control unit 10, having obtained the image data and the audio dataoutput from the video/audio signal input unit 16, controls the displayunit 11 so as to bring up on display at the display unit 11 a screenimage based upon the video data and also enables the audio output unit13 to output sound based upon the audio data by controlling the audiooutput unit 13.

On the other hand, the control unit 20 in the portable terminal 2 isconstituted with a microprocessor, various peripheral circuits, a RAM, aROM and the like, as is the control unit 10 in the in-vehicle device 1,and executes various types of processing based upon a control programrecorded in the memory unit 24.

As explained earlier, the display unit 21 is a touch panel-type displaymonitor. The operation unit 22 is used for purposes of user inputoperation detection. It is to be noted that while the display unit 21and the operation unit 22 are shown as separate structural elements inFIG. 2, the display unit 21 and the operation unit 22 are, in fact,integrated together to constitute a touch panel, as is the display unit11 explained earlier. In the alternative configuration describedearlier, which includes operation switches disposed at the portableterminal 2, the operation switches correspond to the operation unit 22.Details of a user input operation performed at the operation unit 22 areoutput to the control unit 20 and are reflected in the processingexecuted by the control unit 20.

The audio output unit 23, which includes an amplifier, a speaker and thelike, is capable of outputting various types of sound under controlexecuted by the control unit 20. During a phone conversation carried outvia the portable terminal 2, for instance, the voice of the other partyis output from the audio output unit 23.

In the memory unit 24, which is a non-volatile data storage devicesimilar to the memory unit 14 in the in-vehicle device 1, various typesof data to be used in the processing executed by the control unit 20 arestored. Various application programs (hereafter simply referred to asapplications), acquired by the user in advance, are also stored in thememory unit 24. The user is able to select a specific application amongthe various applications stored in the memory unit 24 and have itexecuted by the control unit 20. In this manner, various functions canbe fulfilled in the portable terminal 2.

As does the short-range wireless communication interface unit 15 in thein-vehicle device 1, the short-range wireless communication interfaceunit 25 executes wireless interface processing in compliance with thepredetermined communication standard. Namely, information communicationbetween the in-vehicle device 1 and the portable terminal 2 is achievedas the short-range wireless communication interface unit 15 and theshort-range wireless communication interface unit 25 exchangeinformation with each other through wireless communication.

The video/audio signal output unit 26 converts an image (video image)and sound generated by the control unit 20 to a video signal and anaudio signal in compliance with a predetermined communication standardsuch as HDMI and outputs the signals resulting from the conversion tothe in-vehicle device 1 via the video/audio cable 3. As the video signaland the audio signal are input at the video/audio signal input unit 16in the in-vehicle device 1, an image (a screen) identical to thatbrought up on display at the display unit 21 in the portable terminal 2is brought up on display at the display unit 11 in the in-vehicle device1 and sound identical to that output from the audio output unit 23 inthe portable terminal 2 is also output from the audio output unit 13 inthe in-vehicle device 1. The term “video mirroring” is often used torefer to this function.

The wireless communication unit 27 performs wireless communication toconnect the portable terminal 2 with another portable terminal or aserver via a wireless communication network (not shown). Through thewireless communication carried out via the wireless communication unit27, the portable terminal 2 is able to perform a telephone conversationwith another portable terminal, download a desired application from aserver, and the like. It is to be noted that the wireless communicationnetwork enabling wireless communication carried out via the wirelesscommunication unit 27 may be, for instance, a portable telephone networkor the Internet, connection with which can be established via a wirelessLAN.

The GPS receiver unit 28 receives GPS signals transmitted from GPSsatellites and outputs the GPS signals thus received to the control unit20. Each GPS signal carries, as information that can be used todetermine the current position of the portable terminal 2 and thecurrent time, information indicating the position of the GPS satellitehaving transmitted the particular GPS signal and the transmission timepoint. Thus, the current position and the current time can be calculatedin the control unit 20 based upon information carried in GPS signalsreceived from equal to or more than a predetermined minimum number ofGPS satellites.

Next, a coordinated function achieved by the in-vehicle device 1 and theportable terminal 2 in this in-vehicle information system will bedescribed. A coordinated function achieved by the in-vehicle device 1and the portable terminal 2 is available in the in-vehicle informationsystem. When the coordinated function is in effect, an image and soundcorresponding to a specific application among the various applications,executed at the portable terminal 2 connected with the in-vehicle device1 can be displayed and output at the in-vehicle device 1. In addition,contents of a user operation performed at the in-vehicle device 1 can bereflected in the operation of the application being executed at theportable terminal 2.

For instance, navigation processing may be executed so as to guide thevehicle to a destination by executing a navigation application at theportable terminal 2. In the navigation processing, a map screen imagewith a map of an area around of the current position drawn therein iscreated at the portable terminal 2 and image information indicating themap screen image is output, by using the video signal explained earlier,to the video/audio signal input unit 16 from the video/audio signaloutput unit 26 via the video/audio cable 3. Through this processing, themap screen image is transmitted from the portable terminal 2 to thein-vehicle device 1 so as to bring up the map screen image of the areaaround the current position at the display unit 11 in the in-vehicledevice 1. In addition, as the user sets a destination through a useroperation performed at the operation unit 12 in the in-vehicle device 1or at the operation unit 22 in the portable terminal 2, a search for arecommended route from the current vehicle position designated as adeparture point to the destination having been set is executed at theportable terminal 2. As the vehicle approaches a guidance requiringpoint on the recommended route, audio guidance data indicating thedirection along which the vehicle is to advance at the particularguidance requiring point are transmitted from the portable terminal 2 tothe in-vehicle device 1. As a result, audio guidance can be output fromthe audio output unit 13 in the in-vehicle device 1. It is to be notedthat specific signals may be output from the portable terminal 2 to thein-vehicle device 1, each in correspondence to the timing of the audioguidance data output start and the timing of the audio guidance dataoutput end. Through these measures, it can be ensured that the user willbe able to hear the audio guidance clearly, even if other audio data arebeing output via the radio, the CD player or the like in the in-vehicledevice 1, by lowering the sound volume for the other audio output duringthe audio guidance output. As described above, the in-vehicle device 1provides the user with guidance information so that the user is able todrive the vehicle to the destination without getting lost by displayinga map image at the display unit 11 and outputting audio guidance via theaudio output unit 13.

It is to be noted that various types of data including map data neededby the portable terminal 2 when executing the navigation application maybe stored in advance in the memory unit 24 in the portable terminal 2.As an alternative, only data that are absolutely necessary may be storedin the memory unit 24 and in such a case, the portable terminal 2 shouldbe able to connect with a specific server via the wireless communicationunit 27 in order to obtain necessary additional data each time itexecutes the navigation application.

At the portable terminal 2, an application selected by the user, amongthe plurality of applications including a navigation application such asthat described above, is executed. The user is able to select a desiredapplication he wishes to be executed at the portable terminal 2 byoperating the operation unit 22 in a menu screen on display at thedisplay unit 21 in the portable terminal 2. In the menu screen, forinstance, icons that represent applications for which a coordinatedfunction is available are brought up on display in juxtaposition. Whenthe user selects any one of the icons by operating a touch panel or thelike in the menu screen, an application in correspondence to theselected icon is executed at the portable terminal 2.

In addition, the portable terminal 2 transmits a menu screen image inthe form of a video signal provided from the video/audio signal outputunit 26, to the in-vehicle device 1. Based upon the video signaltransmitted from the portable terminal 2, the in-vehicle device 1 bringsup the menu screen on display at the display unit 11. As the userselects a desired application in this menu screen by a touch operationto the operation unit 12, which is a part of a touch panel, operationinformation corresponding to the touch operation is transmitted via theshort-range wireless communication interface unit 15 from the in-vehicledevice 1 to the portable terminal 2.

The operation information transmitted from the in-vehicle device 1 asdescribed above is received at the short-range wireless communicationinterface unit 25 in the portable terminal 2 and the operationinformation thus received is then output to the control unit 20. Basedupon the operation information received as described above, the controlunit 20 identifies the application selected by the user at thein-vehicle device 1 and executes the selected application. Through thisprocess, the user is able to select a desired application in the menuscreen on display at the in-vehicle device 1, just as he is able toselect a desired application in the menu screen brought up on display atthe display unit 21 at the portable terminal 2, and have the selectedapplication executed in the portable terminal 2.

It is to be noted that the control unit 20 is able to execute eachapplication either in the foreground or in the background. Anyapplication being executed in the foreground is designated as a subjectapplication for image display and operation input both at the in-vehicledevice 1 and at the portable terminal 2. While the control unit 20 doesexecute the corresponding processing for an application running in thebackground, the application is not a subject application for imagedisplay or operation input at the in-vehicle device 1 or the portableterminal 2. However, sound originating from the application beingexecuted in the background may be output.

In order to enable a coordinated function such as that described aboveachieved by connecting the in-vehicle device 1 to the portable terminal2, an application referred to as an application manager is installed inadvance and stored in the memory unit 24 at the portable terminal 2.Namely, a plurality of applications including the application manager isstored in the memory unit 24. As the portable terminal 2 becomesconnected to the in-vehicle device 1, the application manager is readout from the memory unit 24 and is executed by the control unit 20.

FIG. 3 is a diagram schematically illustrating the software structureadopted in the portable terminal 2. An application manager 201 in FIG. 3includes a sub-application Ma and a sub-application Ms.

The sub-application Ma has a launch function for starting upapplications other than the application manager 201 itself and aregulatory information acquisition function for obtaining regulatoryinformation for each application. The control unit 20 is able to usethese functions by executing the sub-application Ma in the foreground.For instance, the launch function is used to call up another applicationand have that application executed in the foreground by the control unit20, instead of the sub-application Ma. In addition, the regulatoryinformation acquisition function is used to obtain the regulatoryinformation indicating contents of action regulation to be imposed oneach application among the plurality of applications stored in thememory unit 24 while the vehicle is in a traveling state from the memoryunit 24, an external server or the like.

The sub-application Ms has a communication function to be engaged whenconnecting the portable terminal 2 to the in-vehicle device 1 and anaction regulation function for imposing action regulation while thevehicle is in the traveling state. The control unit 20 is able to usethese functions by executing the sub-application Ms in the background.For instance, the communication function enables execution ofcommunication processing so as to allow the information terminal 2 andthe in-vehicle device 1 to exchange information needed for coordinatedoperations. In addition, the action regulation function is used todetermine the contents of the action regulation to be imposed on theapplication being executed in the foreground while the vehicle is in atraveling state by referencing the regulatory information obtainedthrough the regulatory information acquisition function of thesub-application Ma described earlier. Restriction information indicatingthe determination results is transmitted from the portable terminal 2 tothe in-vehicle device 1 through the communication function and isutilized in the in-vehicle device 1 when imposing action restrictionwhile the vehicle is in a traveling state.

As explained above, the application manager 201 is configured with twoseparate sub-applications, i.e., the sub-application Ma executed by thecontrol unit 20 in the foreground and the sub-application Ms executed bythe control unit 20 in the background. By adopting this structure,optimization of function assignments within the application manager 201is achieved so as to assign functions best suited for foregroundexecution as foreground functions and assign functions best suited forbackground execution as background functions.

The application manager 201 calls up a given application among variousapplications 202 through the launch function of the sub-application Ma.The application thus called up is then executed by the control unit 20in the foreground instead of the sub-application Ma. It is to be notedthat the following description will be given in reference to FIG. 3 byassuming that application A is being executed.

An OS (operating system) 203 is a software program used to manage theoverall operations of the portable terminal 2. When the portableterminal 2 is connected to the in-vehicle device 1, the OS 203 acts as ago-between for information output from the sub-application Ms executedby the control unit 20 in the background and input to an SPP (SerialPort Profile) 204 and an HID (Human Interface Device) profile 205 andvice versa. The SPP 204 and the HID profile 205 are drivers used inshort-range wireless communication carried out between the in-vehicledevice 1 and the portable terminal 2. These drivers are standardized aspart of the standard used in compliance with Bluetooth.

The SPP 204 executes processing for transmitting the restrictioninformation indicating the results of the determination made with regardto the action regulation contents through the action regulation functionof the sub-application Ms, and processing for receiving travelinginformation transmitted from the in-vehicle device 1 based upon thevehicle traveling state. The HID profile 205 executes processing forreceiving operation information output in correspondence to contents ofthe user operation performed at the in-vehicle device 1. The contents ofthe various types of information received by the SPP 204 and the HIDprofile 205 are output to the sub-application Ms via the OS 203 and arethen transferred to the application, the execution of which is underway,through the communication function of the sub-application Ms. It is tobe noted that the transmission/reception of the various types ofinformation is enabled through wireless communication carried out by theshort-range wireless communication interface unit 15 at the in-vehicledevice 1 and the short-range wireless communication interface unit 25 atthe portable terminal 2.

When the sub-application Ma is currently executed in the foreground bythe control unit 20, the sub-application Ma generates a menu screenimage that will allow the user to select an application he wishes tohave executed by the launch function explained earlier. If, on the otherhand, an application A is currently being executed in the foreground bythe control unit 20, the application A generates a specific image andsound by utilizing, as needed, the traveling information or theoperation information transferred from the sub-application Ms. The imageand the sound are then stored into an image/audio memory 206 on atemporary basis, before they are output to an HDMI driver 207.

The HDMI driver 207 executes processing for converting an image andsound generated by the sub-application Ma, application A or the like toa video signal and an audio signal through a method in compliance withthe HDMI standard. The video signal and the audio signal resulting fromthe conversion are then output by the video/audio signal output unit 26to in-vehicle device 1 via the video/audio cable 3.

The portable terminal 2 has a software structure as described above. Itis to be noted that the software structure may be achieved by using, forinstance, the Android operating system. In the software configured inthe Android operating system, the sub-application Ma will be executed inthe “Activity” thread and the sub-application Ma will be executed in the“Service” thread so as to enable execution of the sub-application Ma inthe foreground by the control unit 20 concurrently with thesub-application Ms, being executed in the background.

Next, operations that will occur when a touch operation is performed bythe user at the in-vehicle device 1 are explained in detail. Asdescribed earlier, when a touch operation is performed by the user onthe screen that is brought up on display on the display unit 11 of thein-vehicle device 1, operation information corresponding to the touchposition specified by the touch operation is transmitted from thein-vehicle device 1 to the portable terminal 2. On this occasion, thein-vehicle device 1 uses a communication format for a mouse, called HIDpacket, for transmitting the operation information to the portableterminal 2.

It is to be noted that the HID packet, which is a communication formatfor a mouse used in Bluetooth, are generally used when a mouse is usedas an input device of the portable terminal 2. When the mouse is used,according to the HID packet, displacement quantity informationcorresponding to the movement of the mouse and button operationinformation corresponding to various types of button operations, forinstance, a click operation in a predetermined format can be transmittedfrom the mouse to the portable terminal 2 at predetermined timeintervals. Upon receipt of these pieces of information transmitted fromthe mouse via the HID packet, the portable terminal 2 performs receptionprocessing through the HID profile 205 shown in FIG. 3 and the OS 203reads the received information to detect the movement of the mouse andcauses the cursor on the screen to be displaced in correspondence to thedetected movement of the mouse. In addition, when a button operationsuch as click or the like is performed, the OS 203 reads the content ofthe operation that is performed by the user on the screen based upon thecontent of the screen corresponding to the cursor position at that timeand the content of the button operation. At the control unit 20, anapplication corresponding to the icon specified on the menu screen incorrespondence to the content of the operation of the mouse read by theOS 203 is started up or the processing corresponding to the operationspecified on the screen of the application being executed is executed.

In the in-vehicle information system according to the presentembodiment, the operation information corresponding to the touchoperation at the in-vehicle device 1 is transmitted from the in-vehicledevice 1 to the portable terminal 2 by using the HID packet as describedabove. Through this measure, the content of touch operation can berecognized by the OS 203 at the portable terminal 2, similarly to thecase in which a mouse is used.

FIG. 4( a) and FIG. 4( b) are diagrams illustrating the operations ofthe in-vehicle device 1 and the portable terminal 2, respectively, whenthe touch operation is performed at the in-vehicle device 1. FIG. 4( a)illustrates the condition before the touch operation. In FIG. 4( a),arrow-shaped cursors 40 brought up on display at respective right lowerends of the outer peripheries of the display units 11 and 21,respectively, have each a tip portion of the arrow, a left upper end ofwhich serves as a point of indication. In other words, before the touchoperation, the cursor 40 on the screen indicates a reference position 41located at a position accounting for the right lower end of the displayunit 11 in the in-vehicle device 1 or a reference position 42 located ata position accounting for the right lower end of the display unit 21. Itis to be noted that the most part of the cursor 40 is located outsidethe range of each screen display at the display units 11 and 21 andactually, it is not brought up on display. Therefore, in FIG. 4( a), thedisplay position of the cursor 40 is shown in broken line.

Before the touch operation, the portable terminal 2 brings up at thedisplay unit 21 a screen including the cursor 40 brought up on displayat the reference position 42 and at the same time outputs videoinformation of this screen to the in-vehicle device 1 via thevideo/audio cable 3 by using the video signal described above. When thein-vehicle device 1 receives the video information from the portableterminal 2, the in-vehicle device 1 brings up on display at the displayunit 11 the same screen as that which is brought up on display at thedisplay unit 21 of the portable terminal 2 based upon the videoinformation. Through these measures, as shown in FIG. 4( a), the screenincluding the cursor 40 brought up on display at the reference position41 is displayed at the display unit 11.

Here, as shown in FIG. 4( a), it is assumed that the resolution of thedisplay unit 11 in the X direction (transverse direction) is expressedas Xv, the resolution of the display unit 11 in the Y direction(longitudinal direction) is expressed as Yv. It is also assumed that theresolution of the display unit 21 in the X direction (transversedirection) is expressed as Xs and the resolution of the display unit 21in the Y direction (longitudinal direction) is expressed as Ys. Theinformation about these resolutions is stored in advance in the memoryunits 14 and 24 at the in-vehicle device 1 and the portable terminal 2,respectively.

FIG. 4( b) shows the condition when a touch operation is performed. Itis assumed that as shown in FIG. 4( b), the touch operation is performedby the user to the in-vehicle device 1 to specify a touch position 43.When the in-vehicle device 1 detects this touch operation, it calculatesa displacement quantity Px in the X direction and a displacementquantity Py in the Y direction, as quantities of displacement rangingfrom the reference position 41, which was indicated by the cursor 40before the touch operation, to the touch position 43. The displacementquantities Px and Py can be calculated by, for instance, counting thenumber of pixels existing between the reference position 41 and thetouch position 43 in the X direction and in the Y direction,respectively. In this case, when it is assumed that the left upper endof the display unit 11 is defined to be a start point, the rightwardorientation is defined to be a positive orientation of the X direction,and the downward orientation is defined to be a positive orientation ofthe Y direction, both the displacement quantities Px and Py between thereference position 41 and the touch position 43 are obtained as negativevalues.

After the displacement quantity Px in the X direction and thedisplacement quantity Py in the Y direction are calculated as describedabove, the in-vehicle device 1 subsequently converts the displacementquantities Px and Py into displacement quantities Qx and Qy,respectively, on the display unit 21 of the portable terminal 2. Thisconversion of displacement quantities is performed based upon theresolutions Xv, Yv and the resolutions Xx, Ys described above accordingto the following expression (1).

Qx=PxX(Xs/Xv), Qy=PyX(Ys/Yv)  (1)

In the expression (1) above, Xs/Xv and Ys/Yv are conversion coefficientsused when the displacement quantities Px and Py are converted into thedisplacement quantities Qx and Qy, respectively. The in-vehicle device 1is enabled to obtain these conversion coefficients by obtaining theresolutions Xs and Ys of the display unit 21 from the portable terminal2 in advance.

Once the displacement quantities Qx and Qy after the conversion arecalculated, the in-vehicle device 1 transmits displacement quantityinformation indicating the displacement quantities Qx and Qy to theportable terminal 2 according to the HID packet.

When the portable terminal 2 receives displacement quantity informationindicating the displacement quantities Qx and Qy from the in-vehicledevice 1, it causes the displacement quantities Qx and Qy to be read bythe OS 203. Then, as shown in FIG. 4( b), it brings up on display, atthe display unit 21, the cursor 40 as being displaced by thedisplacement quantities Qx and Qy from the reference position 42 locatedat the right lower end at the display unit 21 and at the same timeoutputs video information of the screen including the cursor 40 thusdisplaced to the in-vehicle device 1 via the video/audio cable 3 byusing the video signal described above.

When the in-vehicle device 1 receives the video information about thescreen including the cursor 40 after the displacement from the portableterminal 2, it brings up on display the same screen as that brought upon display at the display unit 21 of the portable terminal 2 at thedisplay unit 11 based upon the received video information. As a result,the screen in which the cursor 40 specifies the touch position 43 isbrought up on display at the display unit 11 as shown in FIG. 4( b).

After having transmitted the displacement quantity informationindicating the displacement quantities Qx and Qy, the in-vehicle device1 transmits displacement quantity information indicating that thedisplacement quantity is 0 to the portable terminal 2 through the HIDpacket. On this occasion, additionally, button operation informationindicating that a predetermined button operation, for instance, a leftclick operation has been performed is transmitted together.

Upon receipt of the displacement quantity information and the buttonoperation information from the in-vehicle device 1, the portableterminal 2 causes the content of the button operation that is indicatedby the button operation information to be read by the OS 203. Then, forinstance, startup of the application corresponding to the icon at thetouch position 43 is performed in correspondence to the content of thescreen brought up on display at the display position, that is, the touchposition 43, of the cursor 40 after the displacement and the content ofthe button operation. In the manner as that explained earlier, theprocessing in correspondence to the touch operation performed by theuser to the in-vehicle device 1 can be executed at the portable terminal2.

It is to be noted that the operation at the time of the touch operationexplained above is an example of a single tap operation, which isspecified by the user's tapping a given touch position on the touchpanel once. In this case, the in-vehicle device 1 transmits to theportable terminal 2 the same operation information as that transmittedwhen a click operation is performed with a mouse. As another way, theuser may perform various operations; for instance, a double tapoperation in which the touch position is specified by tapping twice, aflick operation in which along with the touch operation, the touchposition is displaced in a predetermined direction, a press-and-holdoperation in which the same touch position is continued to be touchedfor a predetermined period of time or longer, and so on may be performedto the in-vehicle device 1. The in-vehicle device 1 identifies types ofthe touch operations, one from another, and transmits to the portableterminal 2 different operation information in correspondence to therespective types of the touch operations.

Specifically, when the double tap operation is performed, the in-vehicledevice 1 transmits displacement quantity information indicating thedisplacement quantities Qx and Qy after conversion in correspondence tothe displacement quantities Px and Py and thereafter transmits twice thebutton operation information indicating a predetermined button operationalong with the displacement quantity information indicating that thedisplacement quantity is 0. In other words, the in-vehicle device 1transmits to the portable terminal 2 the same operation information asthat transmitted when the double click operation is performed with amouse.

On the other hand, when the flick operation is performed, the in-vehicledevice 1 transmits the displacement quantity information indicating thedisplacement quantities Qx and Qy after conversion in correspondence tothe displacement quantities Px and Py and thereafter calculates thedisplacement quantity of the touch position during the flick operationat predetermined time intervals. Then, the in-vehicle device 1 converts,according to the expression (1) described above, the displacementquantity of the touch position during the flick operation into thedisplacement quantity on the display unit 21 of the portable terminal 2and transmits the displacement quantity information indicating thedisplacement quantity after the conversion along with the buttonoperation information indicating a predetermined button operation. Afterthe flick operation is completed, the in-vehicle device 1 finallytransmits the displacement quantity information indicating that thedisplacement quantity is 0. In other words, the in-vehicle device 1transmits to the portable terminal 2 the same operation information asthat transmitted when the drag-and-drop operation is performed with amouse.

In addition, when the press-and-hold operation is performed, thein-vehicle device 1 transmits the displacement quantity informationindicating the displacement quantities Qx and Qy after conversion incorrespondence to the displacement quantities Px and Py, respectively,and thereafter continually transmits the button operation informationindicating a predetermined button operation along with the displacementquantity information indicating that the displacement quantity is 0during the press-and-hold operation. That is, the in-vehicle device 1transmits to the portable terminal 2 the same operation information asthat transmitted when press and hold operation is performed with amouse.

As the portable terminal 2 receives the operation information incorrespondence to the type of the touch operation as explained earlierfrom the in-vehicle device 1, the portable terminal 2 can identify thetype of the touch operation performed by the user to the in-vehicledevice 1 and execute the processing corresponding to the touchoperation.

FIG. 5 presents a flowchart illustrating the processing executed at thein-vehicle device 1 when a touch operation is performed. The processingillustrated in this flowchart is to be executed by the control unit 10once communication is established between the in-vehicle device 1 andthe portable terminal 2.

In step S10, the control unit 10 receives the resolution information ofthe display unit 21 transmitted from the portable terminal 2 by usingthe short-range wireless communication interface unit 15. In this step,as described earlier, the control unit 10 receives the resolution Xs inthe X direction of the display unit 21 and the resolution Ys in the Ydirection of the display unit 21 from the portable terminal 2.

In step S20, the control unit 10 calculates a conversion coefficient incorrespondence to a ratio of the resolution of the display unit 21 tothe resolution of the display unit 11 based upon the resolutioninformation of the display unit 21 received from the portable terminal 2in step S10 and the resolution of the display unit 11 stored in thememory unit 14 in advance. Here, as shown in the expression (1)described above, the resolutions Xs and Ys of the display unit 21 shownin the resolution information received in step S10 are divided by theresolution Xv of the display device 11 in the X direction and theresolution Yv of the display unit 11 in the Y direction, respectively,to calculate a conversion coefficient in the X direction, Xx/Xv and aconversion coefficient in the Y direction, Ys/Yv, respectively.

In step S30, the control unit 10 makes a decision as to whether or not atouch operation has been performed by the user to the operation unit 12,which is a touch panel switch that is integrally constituted with thedisplay unit 11. If a touch operation has been performed, the controlunit 10 detects the touch operation and the operation proceeds to stepS40.

In step S40, the control 10 calculates displacement quantities from thedisplay position of the cursor when the touch operation was detected instep S30 to the touch position specified by that touch operation. Here,as described earlier, the control unit 10 calculates the displacementquantity in the X direction, Px, and the displacement quantity in the Ydirection, Py, as the displacement quantities from the referenceposition 41 at the right lower end to the touch position 43.

In step S50, the control unit 10 converts the displacement quantitiescalculated in step S40 into displacement quantities on the display unit21 of the portable terminal 2. Here, the control unit 10 converts thedisplacement quantity in the X direction, Px, and the displacementquantity in the Y direction, Py, calculated relative to the display unit11, the displacement quantities in the X direction, Qx, and thedisplacement quantity in the Y direction, Qy on the display unit 21according to the expression (1) described above by using the conversioncoefficient in the X direction, Xs/Xv, and the conversion coefficient inthe Y direction, Ys/Yv, that have been calculated in step S20.

In step S60, the control unit 10 transmits to the portable terminal 2the displacement quantity information indicating the displacementquantity converted in step S50 by using the short-range wirelesscommunication interface unit 15. As a result, displacement quantityinformation in correspondence to the displacement quantity from thedisplay position of the cursor at the time of the touch operation to thedetected touch position is transmitted from the in-vehicle device 1 tothe portable terminal 2. Transmission of the displacement quantityinformation is performed by using the HID packet, which is acommunication format for a mouse as described above. The displacementquantity information transmitted from the in-vehicle device 1 isreceived by the short-range wireless communication interface unit 25 inthe portable terminal 2 and read by the OS 203. As a result, the cursorposition on the screen is displaced at the portable terminal 2 and thevideo information including the cursor after the displacement istransmitted from the portable terminal 2 to the in-vehicle device 1 andbrought up on display on the display unit 11.

In step S70, the control unit 10 makes a decision as to whether or notthe touch operation detected in step S30 is going on. If the touchoperation is completed within a predetermined period of time, it isdecided that the touch operation is not going on and the operationproceeds to step S80, whereas if it is decided that the touch operationis going on, the operation proceeds to step S130.

When the operation proceeds from step S70 to step S80, the control unit10 sets the displacement quantity in the displacement quantityinformation to 0 in step S80. Here, the displacement quantity is set to0 for both the X direction and the Y direction. Subsequently, in stepS90, the control unit 10 transmits the displacement quantity informationindicating that the displacement quantity set in step S80 is 0 and thebutton operation information indicating that the predetermined buttonoperation such as a left click operation has been performed to theportable terminal 2 via the short-range wireless communication interfaceunit 15.

In step S100, the control unit 10 makes a decision as to whether or notanother touch operation is performed to the operation unit 12, which isa touch panel switch that is integrally constituted with the displayunit 11, subsequent to the touch operation detected in step S30. Iftouch operations are not performed successively, that is, a single tapoperation is performed, the operation proceeds to step S110. As aresult, if the single tap operation has been performed, the displacementquantity information is transmitted in step S60 and thereafter theprocessing in step S90 is executed only once, in which along with thedisplacement quantity information indicating that the displacementquantity is 0, the button operation information is transmitted from thein-vehicle device 1 to the portable terminal 2.

On the other hand, if a decision is made that the touch operations havebeen performed successively in step S100, the operation returns to stepS90 and the processing in step S90 is repeatedly executed the number oftimes in which the touch operations occurred. As a result, if the doubletap operation has been preformed, the displacement quantity informationis transmitted in step S60 and thereafter, the processing in step S90 isexecuted twice, so that along with the displacement quantity informationindicating that the displacement quantity is 0, the button operationinformation is transmitted twice from the in-vehicle device 1 to theportable terminal 2. It is to be noted that if three or more touchoperations have been performed successively, the third or subsequenttouch operations may be invalidated in order to treat the three or moretouch operations as a double tap operation. As an alternative, aconfiguration may be adopted in which the displacement quantityinformation and the button operation information are transmitted threeor more times successively by repeatedly executing the processing instep S90 as many times as that in which the touch operations have beenperformed.

In step S110, the control unit 10 sets a predetermined maximal value ofdisplacement quantity in the displacement quantity information. Here, apredetermined maximal displacement quantity is set for both the X and Ydirections. Subsequently, in step S120, the control unit 10 transmitsthe displacement quantity information indicating the maximaldisplacement quantity set in step S110 to the portable terminal 2 viathe short-range wireless communication interface unit 15. If theprocessing in step S120 is executed, the operation returns to step S30and the processing as described above is repeated.

The displacement quantity information indicating the maximaldisplacement quantity transmitted in step S120 is received by theshort-range wireless communication interface unit 25 at the portableterminal 2 and read by the OS 203. Then, the display position of thecursor on the screen at the display unit 21 is displaced to maximalpositions in the X and Y positions, that is, to the reference position42 at the right lower end shown in FIGS. 4( a) and 4(b) regardless ofthe touch position specified by lastly performed touch operationrelative to the in-vehicle device 1. As the video information incorrespondence to the screen after the displacement of the cursor istransmitted from the portable terminal 2 to the in-vehicle device 1 andbrought up on display at the display unit 11, the display position ofthe cursor on the screen at the display unit 11 in the in-vehicle device1 is returned back to the original reference position 41.

On the other hand, if the operation proceeds from step S70 to step S130,the control unit 10 in step S130 makes a decision as to whether or notthe touch position has been changed by the touch operation detected instep S30. If it is decided that the touch position has been changed, theoperation proceeds to step S140. If, on the other hand, it is decidedthat the touch position has not been changed, the operation proceeds tostep S200.

If the operation proceeds from step S130 to step S140, the control unit10 in step S140 calculates the displacement quantity from the last touchposition to the present touch position. Subsequently, in step S150, thecontrol unit 10 converts the displacement quantity calculated in stepS140 to displacement quantity on the display unit 21 of the portableterminal 2.

In step S160, the control unit 10 transmits the displacement quantityinformation indicating the displacement quantity converted in step S150and the button operation information indicating that the predeterminedbutton operation such as a left click operation has been performed tothe portable terminal 2 via the short-range wireless communicationinterface unit 15. As a result, when the touch operation is going onwhile the touch position is being changed, the displacement quantityinformation indicating the displacement quantity in correspondence tothe change in the touch position and the button operation informationare transmitted from the in-vehicle device 1 to the portable terminal 2.The transmitted displacement quantity information here is received bythe short-range wireless communication interface unit 25 in the portableterminal 2 and read by the OS 203. As a result, the cursor position onthe screen at the portable terminal 2 is displaced in correspondence tothe change in touch position at the in-vehicle device 1 and the videoinformation including the cursor after the displacement is transmittedfrom the portable terminal 2 to the in-vehicle device 1 and brought upon display at the display unit 11.

In step S170, the control unit 10 makes a decision as to whether or notthe touch operation is completed. If the touch operation is going on,the operation returns to step S140 and the processing in steps S140through S160 is repeated. Through these measures, if a flick operation,in which the touch position is changed while the touch operation isgoing on, has been performed, and if the touch operation is going onafter the displacement quantity information has been transmitted in stepS60, the displacement quantity information indicating the displacementquantity in correspondence to the change in the touch position and thebutton operation information are transmitted continuously. If the touchoperation is completed, the operation proceeds to step S180.

In step S180, the control unit 10 sets the displacement quantity in thedisplacement quantity information to 0 in the same manner as that instep S80 described above. Subsequently, in step S190, the control unit10 transmits the displacement quantity information indicating that thedisplacement quantity set in step S180 is 0 to the portable terminal 2via the short-range wireless communication interface unit 15. Throughthis measure, if the flick operation has been performed and after thetouch operation has been completed, the displacement quantityinformation indicating that the displacement quantity is 0 istransmitted from the in-vehicle device 1 to the portable terminal 2.

Once the processing in step S190 is executed, the control unit 10 causesthe display position of the cursor on the screen at the display unit 11to be returned back to the original reference position 41 by executingeach processing in steps S110 and S120 described above. Thereafter, theoperation returns to step S30 and the processing as described above isrepeated.

On the other hand, if the operation proceeds from step S130 to S200, thecontrol unit 10 in step S200 sets the displacement quantity in thedisplacement quantity information to 0 in the same manner as that insteps S80 and S180 described above. Subsequently in step S210, thecontrol unit 10 transmits the displacement quantity informationindicating that the displacement quantity set in step S200 is 0 and thebutton operation information indicating that a predetermined buttonoperation such as a left click operation has been performed to theportable terminal 2 via the short-range wireless communication interfaceunit 15.

In step S220, the control unit 10 makes a decision as to whether or notthe touch operation is completed. If the touch operation is going on,the operation returns to step S210 and the processing in step S210 isrepeated. Through this measure, if a press-and-hold operation, in whicha touch operation is going on for a predetermined period of time orlonger with the touch position being unchanged, has been performed andif the touch operation is going on after the displacement quantityinformation has been transmitted in step S60, the button operationinformation along with the displacement quantity information indicatingthat the displacement quantity is 0 is continuously transmitted. If thetouch operation is completed, the control unit 10 causes the displayposition of the cursor on the screen at the display unit 11 to bereturned to the original reference position 41 by executing eachprocessing in steps S110 and S120. Thereafter, the operation returns tostep S30 and the processing described above is repeated.

According to the embodiment of the present invention explained above,operations and advantageous effects as listed in (1) through (13) belowcan be obtained.

(1) The in-vehicle device 1 receives video information about a screenincluding the cursor 40 from the portable terminal 2 via the video/audiosignal input unit 16 and brings up the screen on display at the displayunit 11 in the form of a touch panel based upon the video information.Then, the in-vehicle device 1 detects a touch position specified by atouch operation by the user with the operation unit 12 that constitutesthe touch panel together with the display unit 11. On this occasion,through the processing by the control unit 10, the displacement quantityinformation in correspondence to the displacement quantity from thedisplay position of the cursor 40 to the touch position is transmittedvia the short-range wireless communication interface unit 15 (step S60).Through this measure, when the display screen of the portable terminal 2in which the touch panel operation is adopted is brought up on displayat the in-vehicle device 1, the portable terminal 2 can be operatedthrough the in-vehicle device 1.

(2) The control unit 10 transmits in addition button operationinformation indicating that a predetermined button operation has beenperformed to the portable terminal 2 via the short-range wirelesscommunication interface unit 15 (steps S90, S160, S210). Through thesemeasures, the content of the touch operation can be recognized at theportable terminal 2 with the processing that is the same as that isexecuted when a mouse is used.

(3) If the touch operation is completed within a predetermined period oftime with the touch position being unchanged, the control unit 10transmits the displacement quantity information in correspondence to thedisplacement quantity from the display position of the cursor 40 to thetouch position and thereafter the control unit 10 transmits the buttonoperation information along with the displacement quantity informationindicating that the displacement quantity is 0 (step S90). Aconfiguration is adopted such that if the single tap operation has beenperformed as described above, the same operation information as thattransmitted when the click operation has been performed with a mouse istransmitted from the in-vehicle device 1 to the portable terminal 2.Therefore, at the portable terminal 2, the single tap operation can berecognized as the click operation.

(4) In addition, if the touch operation has been repeated a plurality oftimes with the touch position being unchanged, the control unit 10transmits, in step S60, displacement quantity information incorrespondence to the displacement quantity from the display position ofthe cursor 40 to the touch position and then the processing in step S90is executed a plurality of times, so that along with the displacementquantity information indicating that the displacement quantity is 0, thebutton operation information is transmitted a plurality of times. Inthis manner, a configuration is adopted such that if the double tapoperation has been performed, the same operation information as thattransmitted in case that the double click operation is performed with amouse is transmitted from the in-vehicle device 1 to the portableterminal 2. Therefore, the double tap operation can be recognized as thedouble click operation at the portable terminal 2.

(5) If the touch position has been changed while the touch operation wasgoing on, the control unit 10 transmits, in step S60, displacementquantity information in correspondence to the displacement quantity fromthe display position to the touch position of the cursor 40 and then, ifthe touch operation is going on, it transmits the button operationinformation along with the displacement quantity information indicatingthe displacement quantity in correspondence to the change in the touchposition (step S160). In addition, after the touch operation iscompleted, it transmits the displacement quantity information indicatingthat the displacement quantity is 0 (step S190). In this manner, aconfiguration is adopted such that if the flick operation has beenperformed, the same operation information as that transmitted in casethat the drag-and-drop operation is performed with a mouse istransmitted from the in-vehicle device 1 to the portable terminal 2.Therefore, the flick operation can be recognized as the drag-and-dropoperation at the portable terminal 2.

(6) If the touch operation is continued for a predetermined period oftime or longer with the touch position being unchanged, the control unit10 transmits the displacement quantity information in correspondence tothe displacement quantity from the display position of the cursor 40 tothe touch position and thereafter, if the touch operation is going on,the control unit 10 transmits the button operation information alongwith the displacement quantity information indicating that thedisplacement quantity is 0 (step S210). In this manner, a configurationis adopted such that if the press-and-hold click operation has beenperformed, the same operation information as that transmitted in casethat the press-and-hold click operation is performed with the mousebeing unmoved is transmitted from the in-vehicle device 1 to theportable terminal 2. Therefore, the press-and-hold operation can berecognized as the press-and-hold click operation at the portableterminal 2.

(7) The control unit 10 is configured to transmit displacement quantityinformation by using the HID packet, which is a communication format fora mouse in Bluetooth via the short-range wireless communicationinterface unit 15 in steps S60, S90, S120, S160, S190 and S210. Throughthese measures, the transmission of the displacement quantityinformation can be achieved readily and with certainty by using theexisting communication format.

(8) The in-vehicle device 1 receives video information of the screenincluding cursor 40 from the portable terminal 2 and based upon thereceived video information brings up on display the screen on thedisplay unit 11, which is a touch panel. Then, the in-vehicle device 1detects the touch position specified through the touch operation by theuser with the operation unit 12 that constitutes the touch paneltogether with the display unit 11. In addition, the in-vehicle device 1receives the resolution information relative to the display unit 21 fromthe portable terminal 2 by using the short-range wireless communicationinterface unit 15 through the processing by the control unit 10 (stepS10) and based upon the resolution information and the resolution of thedisplay unit 11 stored in the memory unit 14 in advance, calculatesconversion coefficients Xs/Xv and Ys/Yv that suit the ratio of theresolution of the display unit 21 to the resolution of the display unit11 (step S20). Then, it calculates the displacement quantities Px andPy, which are displacement quantities from the display position of thecursor 40 to the touch position (step S40) and based upon the conversioncoefficients Xs/Xv and Ys/Yv, respectively, calculated in step S20, itconverts the calculated displacement quantities Px and Py to thedisplacement quantities Qx and Qy, respectively, on the display unit 21(step S50). It transmits the displacement quantity informationindicating the thus converted displacement quantities Qx and Qy to theportable terminal 2 (step S60). Through these measures, when the displayscreen of the portable terminal 2 in which touch panel operation isadopted is brought up on display at the in-vehicle device 1, theportable terminal 2 can be appropriately operated through the in-vehicledevice 1 even if the resolution of the display unit 11 at the in-vehicledevice 1 and the resolution of the display unit 21 at the portableterminal 2 are different from each other.

(9) The control unit 10, in step S20, calculates the conversioncoefficients Xs/Xv and Ys/Yv for the transverse direction (X direction)and the longitudinal direction (Y direction), respectively, relative tothe display unit 11. In addition, in step S40, it calculates thedisplacement quantities Px and Py from the display position of thecursor 40 to the touch position relative to the transverse direction (Xdirection) and the longitudinal direction (Y direction), respectively,of the display unit 11. In step S50, it converts the displacementquantity Px relative to the transverse direction of the display unit 11into the displacement quantity Qx in the transverse direction of thedisplay unit 21 by using the conversion coefficient Xs/Xv in thetransverse direction, and also it converts the displacement quantity Pyrelative to the longitudinal direction of the display unit 21 into thedisplacement quantity Qy in the longitudinal direction of the displayunit 21 by using the conversion coefficient Ys/Yv in the longitudinaldirection of the display unit 21. With these measures, the displacementquantities Qx and Qy after the conversion can be accurately obtainedaccording to the resolutions in the transverse direction and thelongitudinal direction of the display unit 11 and the display unit 21,respectively.

(10) The in-vehicle device 1 receives the video information relative toa specific screen including the cursor 40 brought up on display at thepredetermined reference position 42 from the portable terminal 2 via thevideo/audio signal input unit 16 and based upon the video information,it brings up on display the screen, which is the same as the specificscreen and on which the cursor 40 is brought up on display at thepredetermined reference position 41, at the display unit 11 in the formof a touch panel. Then, it causes the touch position specified throughthe touch operation by the user to be detected via the operation unit12, which constitutes the touch panel together with the display unit 11.On this occasion, through the processing by the control unit 10, thedisplacement quantity information in correspondence to the displacementquantity from the display position of the cursor 40 to the touchposition is transmitted to the portable terminal 2 via the short-rangewireless communication interface unit 15 (step S60). Through thesemeasures, the video information relative to the screen showing that thecursor 40 has been displaced to the touch position is received from theportable terminal 2 and the received video information is brought up ondisplay at the display unit 11. In addition, after the touch operationis completed, the displacement quantity information indicating a maximaldisplacement quantity is transmitted to the portable terminal 2 (stepS120) to thereby cause the display position of the cursor 40 to bereturned to the original reference position 41. Through these measures,even when the display screen of the portable terminal 2, in which thetouch panel operation is adopted, is brought up on display at thein-vehicle device 1 and an error in the cursor position after thedisplacement may occur due to a difference in resolution between thedisplay unit 11 at the in-vehicle device 1 and the display unit 21 atthe portable terminal 2, the portable terminal 2 is enabled to beappropriately operated through the in-vehicle device 1 withoutaccumulating the error.

(11) The control unit 10, in step S120, transmits to the portableterminal 2 the displacement quantity information indicating apredetermined maximal displacement quantity for each of the transversedirection (X direction) and the longitudinal direction (Y direction) ofthe display unit 11 after the touch operation is completed to therebycause the display position of the cursor 40 to be returned to theoriginal reference position 41. As a result, the display position of thecursor 40 is enabled to be returned to the reference position 41 withcertainty regardless of the touch position specified through the lasttouch operation.

(12) Since a configuration is adopted in which the point of indicationof the cursor 40 is located at the left upper end thereof and thereference position 41 is located at the right lower end of the displayunit 11, a most part of the cursor 40 is enabled to be located outsidethe screen display of the display unit 11 when no touch operation isperformed, so that the cursor 40 will not be unpleasant to the eye ofthe user.

(13) The in-vehicle device 1 receives an image from the portableterminal 2 through the video/audio input unit 16 and brings up thereceived image on the display unit 11 in the form of a touch panel.Then, the in-vehicle device 1 detects the touch position specifiedthrough the touch operation by the user via the operation unit 12 thatconstitutes the touch panel together with the display unit 11. On thisoccasion, the in-vehicle device 1 transmits, through the processing bythe control unit 10, the displacement quantity information incorrespondence to the displacement quantity from the display position ofthe cursor 40 to the touch position via the short-range wirelesscommunication interface unit 15 (step S60). In response thereto, itreceives the video information transmitted from the portable terminal 2and brings up the received video information on display at the displayunit 11. As a result, in response to the touch operation, it causes thecursor 40 brought up on display at the reference position 41, which islocated at the outer periphery of the display unit 11 as shown in FIG.4( a), to be displaced to the touch position 43 as shown in FIG. 4( b).In addition, after the touch operation is completed, the displacementquantity information indicating the maximal displacement quantity istransmitted to the portable terminal 2 (step S120) to thereby cause thedisplay position of the cursor 40 to be returned to the originalreference position 41. Through these measures, when the display screenof the portable terminal 2, in which the touch panel operation isadopted, is brought up on display at the in-vehicle device 1 and notouch operation is performed, the portable terminal 2 is enabled to beoperated through the in-vehicle device 1 so that the cursor 40 will notbe unpleasant to the eye of the user.

It is to be noted that the embodiment explained earlier relates to anexample in which when no touch operation is performed, the cursor 40 isbrought up on display at the reference positions 41 and 42 each locatedat the right lower ends of the display units 11 and 12, respectively.However, the cursor 40 may be brought up on display at a position otherthan the above reference positions. As far as the outer peripheries ofthe display units 11 and 12 are concerned, the cursor 40 is enabled tobe arranged so that it will not be unpleasant to the eye of the user inthe same manner as that in the embodiment described above even if thecursor 40 is brought up on display at any desired position when no touchoperation is performed. On this occasion, in order to make the cursor 40as less eye-catching as possible, it is preferred to bring it up ondisplay at a position that is defined as a reference position located atthe end of the display unit 11 in an orientation opposite to theorientation of from the central point of the cursor 40 to the point ofindication. In the example shown in FIGS. 4( a) and 4(b), since thepoint of indication of the cursor 40 is present at the left upper end,the reference position 41 is set at the right lower end in anorientation opposite to the orientation toward the left upperorientation, which is directed from the central point of the cursor 40to the point of indication.

In addition, the embodiment explained earlier relates to an example, inwhich after completion of the touch operation in step S120 shown in FIG.5, the displacement quantity information indicating the predeterminedmaximal displacement quantity for each of the transverse direction (Xdirection) and the longitudinal direction (Y direction) of the displayunit 11 is transmitted to the portable terminal 2 to cause the displayposition of the cursor 40 to be returned to the original referenceposition 41. Assuming that the left upper end of the display unit 11 isdefined to be a start point, the right direction is defined to be apositive orientation of the X direction and the lower direction isdefined to be a positive orientation of the Y direction, this measure iseffective for causing the cursor 40 to be returned to the referenceposition 41 at the right lower end of the display unit 11. However, thiswill not be true if the position of the start point and the definitionsof positive orientations of X and Y directions are changed. Forinstance, assuming that the right lower end of the display unit 11 isdefined to be a start point, the left direction is defined to be thepositive orientation of the X direction, and the upper direction isdefined to be the positive orientation of the Y direction, in order tocause the cursor 40 to be returned to the reference position 41 at theright lower end of the display unit 11, it is necessary to transmit thedisplacement quantity information indicating a predetermined minimaldisplacement quantity for each of the X direction and the Y direction ofthe display unit 11. In other words, in step S120, after completion ofthe touch operation, the displacement quantity information indicatingthe predetermined maximal displacement quantity or the predeterminedminimal displacement quantity in correspondence to the position of thestart point and the definitions of the positive orientations of the Xand Y directions, respectively, relative to the transverse direction (Xdirection) and the longitudinal direction (Y direction) may betransmitted to the portable terminal 2 in order to cause the displayposition of the cursor 40 to be returned to the reference position 41.It is to be noted that the same is true when the right lower end of thedisplay 11 is taken as the reference position.

In the embodiment described above, an example has been explained inwhich the transmission of video signals and audio signals from theportable terminal 2 to the in-vehicle device 1 is enabled by connectingthe in-vehicle device 1 and the portable terminal 2 with each other viathe video/audio cable 3. In addition, an example has been explained inwhich the in-vehicle device 1 and the portable terminal 2 communicatewith each other through short-range wireless communication carried outin compliance with a predetermined communication standard such asBluetooth in the embodiment described above. However, the presentinvention may be achieved in conjunction with another communicationmethod or another signal transmission method. For instance, videosignals and audio signals originating from the portable terminal 2 maybe transmitted to the in-vehicle device 1 through wirelesscommunication. In addition, communication between the in-vehicle device1 and the portable terminal 2 may be carried out through wiredcommunication such as USB communication. On this occasion, in steps S60,S90, S120, S160, S190 and S210 in FIG. 5, the displacement quantityinformation and the button operation information can be transmitted fromthe in-vehicle device 1 to the portable terminal 2 by using thecommunication format for USB or the like instead of the HID packetdescribed earlier. Any communication method may be adopted in thepresent invention as long as it allows the in-vehicle device 1 and theportable terminal 2 to exchange necessary signals and information.

In the embodiment described above, various types of vehicle informationoutput from the vehicle other than the vehicle speed signal and theparking signal may also be obtained by the in-vehicle device 1. Suchvehicle information taken into the in-vehicle device 1 may then beutilized in the processing executed in the in-vehicle device 1, or itmay be output from the in-vehicle device 1 to the portable terminal 2and used in the processing executed at the portable terminal 2. Forinstance, a configuration may be adopted in which a start-up conditionin correspondence to such vehicle information may be set in advance foreach application and when the vehicle information indicating a specificstart-up condition is output from the vehicle, the correspondingapplication may be started up automatically at the portable terminal 2.On this occasion, the information indicating the start-up conditions forthe individual applications may be transmitted from the portableterminal 2 to the in-vehicle device 1 and a decision as to whether ornot a start-up condition exists may be made in the in-vehicle device 1based upon the vehicle information. As an alternative, the vehicleinformation may be transmitted from the in-vehicle device 1 to theportable terminal 2 and the decision as to whether or not a start-upcondition exists may be made at the portable terminal 2 based upon thevehicle information. Such use of the vehicle information makes itpossible for the portable terminal 2 to automatically start-up anapplication that, for instance, enables a search for gas stationslocated near the current position when vehicle information, indicatingthat the quantity of remaining fuel in the vehicle has become less thana predetermined quantity, is output from the vehicle.

In the embodiment described above, the example is explained in which atouch panel is adopted as the display unit 11 of the in-vehicle device 1and an image received from the portable terminal 2 is brought up ondisplay at the display unit 11. However, the present invention is notlimited to this example. The present invention can be adopted in anydisplay device as far as it can be connected with an informationterminal and brings up on display an image received from the informationterminal on a touch panel.

The embodiment and variations thereof described above simply representexamples and the present invention is in no way limited to theseexamples as long as the features characterizing the present inventionremain intact. In addition, the embodiment and variations thereofdescribed above may be adopted in any combination to realize acombination of advantages.

The disclosure of the following priority application is hereinincorporated by reference: Japanese Patent Application No. 2012-130929(filed on Jun. 8, 2012).

REFERENCE SIGNS LIST

1: in-vehicle device, 2: portable terminal, 3: video/audio cable, 10:control unit, 11: display unit, 12: operation unit, 13: audio outputunit, 14: memory unit, 15: short-range wireless communication interfaceunit, 16: video/audio signal input unit, 20: control unit, 21: displayunit, 22: operation unit, 23: audio output unit, 24: memory unit, 25:short-range wireless communication interface unit, 26: video/audiosignal output unit, 27: wireless communication unit, 28: GPS receiverunit.

1. A display device that is to be connected with an informationterminal, comprising: a video reception unit that receives videoinformation about a screen including a cursor from the informationterminal; a touch panel that brings up on display the screen based uponthe video information received by the video reception unit and detects atouch position specified by a touch operation; and a transmission unitthat transmits, to the information terminal, displacement quantityinformation corresponding to a quantity of displacement from the displayposition of the cursor to the touch position.
 2. A display deviceaccording to claim 1, wherein the transmission unit additionallytransmits button operation information indicating that a predeterminedbutton operation has been performed to the information terminal.
 3. Adisplay device according to claim 2, wherein if the touch operation iscompleted within a predetermined period of time with the touch positionbeing unchanged, the transmission unit transmits, after havingtransmitted the displacement quantity information in correspondence tothe quantity of displacement from the display position of the cursor tothe touch position, the button operation information along withdisplacement quantity information indicating that the quantity ofdisplacement is
 0. 4. A display device according to claim 2, wherein ifthe touch operation is performed a plurality of times with the touchposition being unchanged, the transmission unit transmits, after havingtransmitted the displacement quantity information in correspondence tothe quantity of displacement from the display position of the cursor tothe touch position, the button operation information a plurality oftimes along with the displacement quantity information indicating thatthe quantity of displacement is
 0. 5. A display device according toclaim 2, wherein if the touch position is changed while the touchoperation is going on, the transmission unit transmits, after havingtransmitted the displacement quantity information in correspondence tothe quantity of displacement from the display position of the cursor tothe touch position, the button operation information along with thedisplacement quantity information indicating a quantity of displacementin correspondence to a change in the touch position while the touchoperation is going on, and after the touch operation is completed,transmits displacement quantity information indicating that the quantityof displacement is
 0. 6. A display device according to claim 2, whereinif the touch operation is going on for a predetermined period of time orlonger with the touch position being unchanged, the transmission unittransmits, after having transmitted the displacement quantityinformation in correspondence to the quantity of displacement from thedisplay position of the cursor to the touch position, the buttonoperation information along with displacement quantity informationindicating that the quantity of displacement is 0 while the touchoperation is going on.
 7. A display device according to claim 1, whereinthe transmission unit transmits the displacement quantity information byusing a communication format for a mouse in Bluetooth or USB.
 8. Adisplay device that is to be connected with an information terminalhaving a display monitor, comprising: a video reception unit thatreceives video information about a screen including a cursor from theinformation terminal; a touch panel that brings up on display the screenbased upon the video information received by the video reception unitand detects a touch position specified by a touch operation; aresolution information reception unit that receives resolutioninformation of the display monitor from the information terminal; aconversion coefficient calculation unit that calculates a conversioncoefficient in correspondence to a ratio of resolution of the displaymonitor to resolution of the touch panel based upon the resolutioninformation received by the resolution information reception unit andresolution of the touch panel that is stored in advance; a displacementquantity calculation unit that calculates a quantity of displacementfrom a display position of the cursor to the touch position; aconversion unit that converts the quantity of displacement calculated bythe displacement quantity calculation unit into a quantity ofdisplacement on the display monitor based upon the conversioncoefficient calculated by the conversion coefficient calculation unit;and a transmission unit that transmits, to the information terminal,displacement quantity information indicating the quantity ofdisplacement converted by the conversion unit.
 9. A display deviceaccording to claim 8, wherein: the conversion coefficient calculationunit calculates the conversion coefficient for a transverse directionand a longitudinal direction, respectively, of the touch panel; thedisplacement quantity calculation unit calculates a quantity ofdisplacement from a displacement position of the cursor to the touchposition for the transverse direction and the longitudinal direction,respectively, of the touch panel; and the conversion unit converts thequantity of displacement for the transverse direction of the touch panelinto a quantity of displacement for a transverse direction on thedisplay monitor by using the conversion coefficient in the transversedirection and also converts the quantity of displacement for thelongitudinal direction of the touch panel into a quantity ofdisplacement for a longitudinal direction on the display monitor byusing the conversion coefficient in the longitudinal direction.
 10. Adisplay device that is to be connected with an information terminal,comprising: a video reception unit that receives video information abouta screen including a cursor brought up on display at a predeterminedreference position from the information terminal; a touch panel thatbrings up on display the screen based upon the video informationreceived by the video reception unit and detects a touch positionspecified by a touch operation; and a transmission unit that transmits,to the information terminal, displacement quantity informationcorresponding to a quantity of displacement from the display position ofthe cursor to the touch position, wherein after completion of the touchoperation, the display position of the cursor is returned to thereference position.
 11. A display device according to claim 10, whereinafter completion of the touch operation, the transmission unit transmitsdisplacement quantity information indicating a predetermined maximalquantity of displacement or a predetermined minimal quantity ofdisplacement for the transverse direction and the longitudinaldirection, respectively, of the touch panel to the information terminal,thereby returning the display position of the cursor to the referenceposition.
 12. A display device according to claim 10, wherein: a pointof indication of the cursor is located at an end of the cursor; and thereference position is located at an end in an orientation opposite to anorientation directed from a center of the cursor to the point ofindication.
 13. A display device according to claim 12, wherein: thepoint of indication of the cursor is located at a left upper end of thecursor; and the reference position is located at a right lower end ofthe touch panel.
 14. A display device that is to be connected with aninformation terminal, comprising: a video reception unit that receivesan image from the information terminal; and a touch panel that brings upon display the image received by the video reception unit and detects atouch position specified by a touch operation, wherein in response tothe touch operation, a cursor brought up on display at an outerperiphery of the touch panel is displaced to the touch position andafter completion of the touch operation, the cursor is returned to anoriginal position.
 15. A display device according to claim 14, whereinafter completion of the touch operation, displacement quantityinformation indicating a predetermined maximal quantity of displacementor a predetermined minimal quantity of displacement for the transversedirection and the longitudinal direction, respectively, of the touchpanel is transmitted to the information terminal, thereby returning thedisplay position of the cursor to the reference position.
 16. A displaydevice according to claim 14, wherein: a point of indication of thecursor is located at a left upper end of the cursor; and the cursor isbrought up on display at a right lower end of the touch panel before thetouch operation is started and after the touch operation is completed.